This disclosure relates generally to image processing techniques. More particularly, but not by way of limitation, it relates to novel techniques for performing raw camera noise reduction.
In photography, different artifacts, e.g., “noise,” can affect the quality of the image. The defining problem of noise reduction is how much structure can be extracted from a noisy image. At the image sensor, noise can be considered to be white (i.e., no frequency dependence) with a signal dependent variance due to shot noise. Noise is largely un-correlated between color channels (R, G, B). At the end of a typical image processing pipeline (e.g., after undergoing noise reduction, demosaicing, white balancing, filtering, color enhancement, and compression in the image signal processor), image noise may be dependent on signal, frequency, illuminant, and light level, and also may be correlated between channels.
The noise reduction in a typical mobile phone camera pipeline is fairly basic. Noise reduction is constrained by the number of delay lines available for the image signal processor, as well as computational limitations. Second, since it typically takes a few years to design, test, and produce an image signal processor, the noise reduction algorithm in use in an actual product is typically a few generations old. The camera pipeline itself may introduce a number of artifacts such as false edges, sprinkles, and black/white pixel clumps that, from a signal point-of-view, do not appear to be noise, but actually appear more like structure. These artifacts can severely degrade image quality in bright light, especially in the sky regions (aka blue-sky noise), but they are especially severe in low-light. One way to mitigate noise, as well as artifacts, is to increase exposure time so that more photons can be accumulated in the image sensor, but this introduces motion blur.
Other traditional approaches to noise reduction involve edge detection and coring. For instance, a Sobel filter may be used to produce edge gradient magnitudes. Using coring, magnitudes below a certain threshold indicate noisy areas of the image and magnitudes above the threshold indicate image structure.
However, these prior art techniques fail to extract and understand image structure in the most optimal way, e.g., often failing to smooth long edges in images properly and/or overly smoothing tight image details, such as text. Hence, what is needed is an improved raw camera noise reduction method that excels at effectively separating meaningful structure from unwanted noise in an image using a novel “alignment mapping” process.